As we are approaching the age of multimedia information, images are displayed increasingly often on the display of computers. In the course of the spread of the internet, techniques for retrieving the desired image information easily and accurately are growing more important. To find texts on the internet, keywords are effective for searching, and are used in practice on search sites. However, when searching for multimedia material, as typified by images, the use of keywords and the automatic extraction of raw information is still in the research stage, and in reality, browsing searches in which many candidates are presented as search results and judged by a human being are the rule. Techniques that make such browsing searches more efficient are becoming increasingly important.
On the other hand, the amount of multimedia data stored by individuals but also in office environments on computers grows year by year, increasing the importance of techniques for finding the desired data and viewing it efficiently. The general procedure for searching image data on the internet or the like by conventional techniques is largely as follows.
First, to restrict the desired data, the user uses a search site restricting candidates by keywords or the like, and limits the search scope somewhat with keywords or the like. Next, using image characteristics, such as the distribution of colors, the search scope is limited a little more. Furthermore, the search scope that has been restricted somewhat by the above restrictions is shown on a display, and the user performs a visual browsing search.
Consequently, it seems that browsing searches for images have the following features.
A first feature is that the number of image data sets to be displayed for browsing searches may be very large, and may go well into the hundreds or thousands. Often, the displayed objects are chosen from an enormous population, and which specimen is chosen depends on the restriction operation prior to the display, and often there is not much time for processing prior to the display. A second feature is that to perform the restriction, the population has to be read into the computer once, so that it can be assumed that the displayed objects are stored in a storage device of the computer on which the search is carried out. The displayed objects seem to have these properties not only for browsing searches, but also when viewing a large number of images managed by the user.
Furthermore, the following conditions are necessary when displaying a large number of images.
A first condition is that the displayed images have sufficient viewability. For this, a viewability to the extent that the user can grasp what is being displayed is sufficient, and a precise display of the details can be carried out separately, so that no overly precise display is required for list display.
A second condition is the perspicuity of the displayed images. Ensuring the perspicuity between displayed images has the advantages that it makes it easy to judge up to which of a large number of images have been visually checked, and that it facilitates the comparison of images.
A third condition is the ability to handle displayed images. Since the number of images is large, it is required that operations such as scrolling and zooming of the displayed images can be carried out sufficiently fast. Furthermore, improving the handling abilities also brings the advantage that it is not exhausting to continue the searching operation over a long time.
Typical approaches for display formats during browsing searches that have been used conventionally are as follows.
A first conventional browsing search format is that of displaying shrunk images (JP S62-248376A). Shrunk images (thumbnail images) of a plurality of images to be displayed are produced, and arranged on the display screen. This format has the advantage that it is easy to grasp which images have been viewed. However, as the number of displayed images becomes large, the individual images need to be small, so that there is the disadvantage that the details of an image become hard to see and difficult to grasp.
A second conventional browsing format is that of displaying the images distributed over several pages. With this format, the displayed images are not shrunk, or a certain shrinking ratio is not undercut, ensuring a certain size of the displayed images, and the displayed images are distributed across a plurality of pages, each holding as many images as fit onto one screen of the display. The pages can be displayed successively. This has the advantage that the individual images are easy to grasp, but the perspicuity is deficient, and it is not easy to judge how many images have been viewed, and there is also the disadvantage that the comparison of displayed images is difficult.
A third conventional browsing format is that of a scrolling display. The images are arranged on a large virtual screen allowing list display, and for the actual process of displaying on the display, a scroll function is provided that makes it possible to display a region of the size of the display as a portion of the large virtual screen. With this format, it is easier to compare with other images than with the page splitting format of the second browsing search format, but it still cannot be said to be sufficient.
A fourth conventional browsing format is that of arranging the images in a virtual 3D space for display. This format arranges the displayed images in a virtual 3D space. A function is provided, with which the viewpoint in the virtual 3D space can be shifted. When the viewpoint approaches individual images, the images are zoomed in. Due to our ability to grasp 3D space, it becomes easy to judge how much has been viewed and to compare with other images, and there is further the advantage that the viewability of individual images can be ensured, and that it is comparatively easy to grasp. A publicly known example is JP H09-259130A, which discloses a technique for arranging information hierarchically to perform browsing searches.
In order to obtain a displayed screen on the display (projection screen) with this conventional format of arranging and displaying in a virtual 3D space, 3D objects are projected onto the display by central projection (perspective projection). As shown in FIG. 22, this central projection connects points on the 3D object surface and points on the display by the following relation. Taking the direction of the line of sight as Z, the display is placed at, for example, a position of Z=1, and the points where the line segments connecting the projection center and one point on the 3D object surface intersects with the display are taken as the correspondence points. For Z, any value can be used except 1, which is the value where the display is placed. When (X, Y, Z) are the coordinates of the points on the 3D object surface, then the coordinates (x, y) of the points on the corresponding display are x=X/Z and y=Y/Z. In principle, a 3D display can be achieved by drawing all pixels on the 3D object surface as points on the display. The display is made of pixels, so that the process that is performed is that of allotting colors based on the above-described correspondence relation for all the pixels on the display. With this central projection, a display screen with a sense of realism can be attained, that is close to the appearance perceived by a human being when viewing from the projection center a 3D object that is placed in the virtual 3D space.
With this conventional technology, if images are placed in the virtual 3D space and displayed from a viewpoint that is set within that space using central projection, then the projection center in FIG. 22 becomes the viewpoint, and a display screen with a sense of realism can be attained, that is close to the appearance of the images placed in the virtual 3D space perceived from position of that viewpoint
Consequently, if the number of displayed images is comparatively large, this format is excellent for displaying in 3D for browsing searches, but this format has the following problems.
A first problem is that for 3D displays the calculation amount increases in comparison to 2D displays, and the display processing takes considerable time. In 3D display processing, in particular when displaying large numbers of images on a display, the display speed drops, making this method difficult to use. To perform browsing searches by shifting the viewpoint, a smooth display is necessary when shifting, but when the display speed drops, it is not possible to perform smooth shifting, which severely impedes the convenience.
A second problem is that the content of individual images during viewpoint shifting becomes hard to grasp. If the viewpoint is shifted within the virtual 3D space, then, when the speed with which the viewpoint is shifted accelerates, the content of images shifting at high speeds becomes hard to grasp. Furthermore, since the viewpoint shifts within the 3D virtual space, the image may be warped due to the relation between the visual field angle and the images, and, if this warping changes due to a shifting of the viewpoint, the content of the images may be difficult to grasp intuitively.
A third problem is that even when a large number of small images is arranged in a virtual 3D space, they are difficult to be viewed as three-dimensional. It is difficult to grasp the stereoscopic depth, and the images appear rather as images of different size that are arranged two-dimensionally. In this case, it makes no sense to arrange them in 3D space, as this rather risks some confusion.
A fourth problem is that repeating the process of shifting the viewpoint may strain the user's eyes, making the process less convenient. For the user to grasp the content of an image that shifts in the virtual 3D space, the eye's viewpoint (focus point) needs to be shifted together with the shifting image, and this shifting of the viewpoint is one cause for strain on the eyes.